Investigating Cue Selection and Placement in Tutorial Discourse

Our goal is to identify the features that predict cue selection and placement in order to devise strategies for automatic text generation. Much previous work in this area has relied on ad hoc methods. Our coding scheme for the exhaustive analysis of discourse allows a systematic evaluation and re nement of hypotheses concerning cues. We report two results based on this analysis: a comparison of the distribution of since and because in our corpus, and the impact of embeddedness on cue selection. Discourse cues play a crucial role in many discourse processing tasks, including plan recognition (Litman and Allen, 1987), anaphora resolution (Grosz and Sidner, 1986), and generation of coherent multisentential texts (Elhadad and McKeown, 1990; Roesner and Stede, 1992; Scott and de Souza, 1990; Zukerman, 1990). Cues are words or phrases such as because, first, although and also that mark structural and semantic relationships between discourse entities. While some speci c issues concerning cue usage have been resolved (e.g., the disambiguation of discourse and sentential cues (Hirschberg and Litman, 1993)), our concern is to identify general strategies of cue selection and placement that can be implemented for automatic text generation. Relevant research in reading comprehension presents a mixed picture (Goldman and Murray, 1992; Lorch, 1989), suggesting that felicitous use of cues improves comprehension and recall, but that indiscriminate use of cues may have detrimental e ects on recall (Millis et al., 1993) and that the bene t of cues may depend on the subjects' reading skill and level of domain knowledge (McNamara et al., In press). However, interpreting the research is problematic because the manipulation of cues both within and across studies has been very unsystematic (Lorch, 1989). While Knott and Dale (1994) use systematic manipulation to identify functional categories of cues, their method does not provide the description of those functions needed for text generation. For the study described here, we developed a coding scheme that supports an exhaustive analysis of a discourse. Our coding scheme, which we call Relational Discouse Analysis (RDA), synthesizes two accounts of discourse structure (Grosz and Sidner, 1986; Mann and Thompson, 1988) that have often been viewed as incompatible. We have applied RDA to our corpus of tutorial explanations, producing an exhaustive analysis of each explanation. By doing such an extensive analysis and representing the results in a database, we are able to identify patterns of cue selection and placement in terms of multiple factors including segment structure and semantic relations. For each cue, we determine the best description of its distribution in the corpus. Further, we are able to formulate and verify more general patterns about the distribution of types of cues in the corpus. The corpus study is part of a methodology for identifying the factors that in uence e ective cue selection and placement. Our analysis scheme is coordinated with a system for automatic generation of texts. Due to this coordination, the results of our analyses of \good texts" can be used as rules that are implemented in the generation system. In turn, texts produced by the generation system provide a means for evaluation and further re nement of our rules for cue selection and placement. Our ultimate goal is to provide a text generation component that can be used in a variety of application systems. In addition, the text generator will provide a tool for the systematic construction of materials for reading comprehension experiments. The study is part of a project to improve the explanation component of a computer system that trains avionics technicians to troubleshoot complex electronic circuitry. The tutoring system gives the student a troubleshooting problem to solve, allows the student to solve the problem with minimal tutor interaction, and then engages the student in a postproblem critiquing session. During this session, the system replays the student's solution step by step, pointing out good aspects of the solution as well as ways in which the solution could be improved. To determine how to build an automated explanation component, we collected protocols of 3 human expert tutors providing explanations during the critiquing session. Because the explanation component we are building interacts with users via text and menus, the student and human tutor were required to communicate in written form. In addition, in order to study e ective explanation, we chose experts who were rated as excellent tutors by their peers, students, and superiors. 1 Relational Discourse Analysis Because the recognition of discourse coherence and structure is complex and dependent on many types of non-linguistic knowledge, determining the way in which cues and other linguistic markers aid that recognition is a di cult problem. The study of cues must begin with descriptive work using intuition and observation to identify the factors a ecting cue usage. Previous research (Hobbs, 1985; Grosz and Sidner, 1986; Schi rin, 1987; Mann and Thompson, 1988; Elhadad and McKeown, 1990) suggests that these factors include structural features of the discourse, intentional and informational relations in that structure, givenness of information in the discourse, and syntactic form of discourse constituents. In order to devise an algorithm for cue selection and placement, we must determine how cue usage is affected by combinations of these factors. The corpus study is intended to enable us to gather this information, and is therefore conducted directly in terms of the factors thought responsible for cue selection and placement. Because it is important to detect the contrast between occurrence and nonoccurrence of cues, the corpus study must be be exhaustive, i.e., it must include all of the factors thought to contribute to cue usage and all of the text must be analyzed. From this study, we are deriving a system of hypotheses about cues. In this section we describe our approach to the analysis of a single speaker's discourse, which we call Relational Discourse Analysis (RDA). Applying RDA to a tutor's explanation is exhaustive, i.e., every word in the explanation belongs to exactly one element in the analysis. All elements of the analysis, from the largest constituents of an explanation to the minimal units, are determined by their function in the discourse. A tutor may o er an explanation in multiple segments, the topmost constituents of the explanation. Multiple segments arise when a tutor's explanation has several steps, e.g., he may enumerate several reasons why the student's action was ine cient, or he may point out the aws in the student's step and then describe a better alternative. Each segment originates with an intention of the speaker; segments are identi ed by looking for sets of clauses that taken together serve a purpose. Segments are internally structured and consist of a core, i.e., that element that most directly expresses the segment purpose, and any number of contributors, the remaining constituents in the segment each of which plays a role in serving the purpose expressed by the core. For each contributor in a segment, we analyze its relation to the core from an intentional perspective, i.e., how it is intended to support the core, and from an informational perspective, i.e., how its content relates to that of the core. Each segment constituent, both core and contributors, may itself be a segment with a core:contributor structure, or may be a simpler functional element. There are three types of simpler functional elements: (1) units, which are descriptions of domain states and actions, (2) matrix elements, which express a mental attitude, a prescription or an evaluation by embedding another element, and (3) relation clusters, which are otherwise like segments except that they have no core:contributor structure. This approach synthesizes ideas which were previously thought incompatible from two theories of discourse structure, the theory proposed by Grosz and Sidner (1986) and Rhetorical Structure Theory (RST) proposed by Mann and Thompson (1988). The idea that the hierarchical segment structure of discourse originates with intentions of the speaker, and thus the de ning feature of a segment is that there be a recognizable segment purpose, is due to Grosz and Sidner. The idea that discourse is hierarchically structured by pairwise relations in which one relatum (the nucleus) is more central to the speaker's purpose is due to Mann and Thompson. Work by Moore and Pollack (1992) modied the RST assumption that these pairwise relations are unique, demonstrating that intentional and informational relations occur simultaneously. Moser and Moore (1993) point out the correspondence between the relation of dominance among intentions in Grosz and Sidner and the nucleussatellite distinction in RST. Because our analysis realizes this relation/distinction in a form di erent from both intention dominance and nuclearity, we have chosen the new terms core and contributor. To illustrate the application of RDA, consider the partial tutor explanation in Figure 1. The purpose of this segment is to inform the student that she made the strategy error of testing inside part3 too soon. The constituent that expresses the purpose, in this case (B), is the core of the segment. The other constituents help to achieve the segment purpose. We analyze the way in which each contributor relates to the core from two perspectives, intentional and informational, as illustrated below. Each constituent may itself be a segment with its own core:contributor structure. For example, (C) is a subsegment whose 1In order to make the example more intelligible to the reader, we replaced references to parts of the circuit with the simple labels part1, part2 and part3. purpose is to give a reason for testing part2 rst, namely that part2 is more susceptible to damage and therefore a more likely source of the circuit fault. The core of this subsegment is (C.2) because it most directly expresses this purpose. The contributor in (C.1) provides a reason for this susceptibility, i.e., that part2 is moved frequently.